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ON METHOD OF STATISTICAL 
DIFFERENTIALS 

by 

Roliitlia Goonatilake^ 

ABSTRACT: The method of statistical differentials, which approximates the mean 
value and variance of transformations of random variables is used in many areas of math- 
ematics. This paper will discuss the conditions under which such an approximation will 
be exact, and also explore their accuracy in terms of error bounds under certain moment 
conditions. 
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I. PRELIMINARIES: 

The method of statistical differentials is a method of approximating the mean (ex- 
pected value), and the variance of transformations of the random variables. The method 
has been presented by Johnson and Kotz [1], Elandt- Johnson and Johnson [2], and Lon- 
don [3]. Frye [4] has given counter examples for the approximation to be exact, and 
also to show that in some cases, this approximation will not hold beyond certain degreed 
polynomials. 

Throughout this paper, we will use the notations adopted by London [3]. Let Y = 
g{X), where X is a random variable such that E[X] and Var[X] exist and are known. 
Other moment conditions will be assumed, depending upon the cases, that are discussed 
from time to time. Let m = E[X]. In order to express Y = g{X) as a Taylor series 
expanded about X = m, assume g{X) is a function possessing derivatives of all order up 
to throughout the interval a < X < b, then there is a value z, with z G (m, X)U{X, m) 
such that 



(X - niY 

g{m) + {X- m)g'{m) + ^ -^g'\m) + 

(^•-"')li/-.(,„) + (^'-"')%M(3). 



n - 1 ! 



The random variable z lies between m and X. Taking expected values of both sides gives, 

\X-mf' 



E[Y] 



E[g{X)] = g{m) + E[X - m]g'{m) + E 



+ E 



{X -m 



(n- 1)! 



g 



{X -m) 



2! 



g"{m) + 
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Since E[X - m] = and E[{X - mf] = Var[X], we have 



E[Y] = + ^ . Var[X] + ^3(^^/V) + 



+ E 



2! 

(X-m)"-i 
(n- 1)! 



3! 



g^''-^\m) + E 



n! 



(1) 



where n is a positive integer greater than 1. It is customary to truncate this series at the 
second term, and to consider 

E[Y] - g{m) + ^ • Var[X] 

as statistical differential approximation. We refer to the approximation (1) as a statistical 
differential approximation. 



II. MOTIVATION: 



We first give an important order notation, which we use time to time throughout this 
paper. The O— notation (read big-oh notation) provides a special way to compare relative 
sizes of functions that is very useful in the analysis of error bounds. The o— notation (read 
small-oh notation is given for completeness of the definition and will not be used in the 
discussion elsewhere. 



Definition 1 Landau Order Notations: 

Let f{x) and g{x) be given functions. Let xq be a fixed point and suppose that g{x) is 
positive and continuous in an open interval about xo, where xq may be finite or infinite. 

1. If there is a constant K such that 



\f{x)\<Kg{x) 

in an open interval about xq, then f{x) = 0(^g{x)^, {x — >• Xg). 
2. Furthermore, if 



lim 44 = 0, 

=^^^0 g[x) 



then f{x) = o(^g{x)y {x Xo). 



Three conditions (i), (ii) and (iii) below, each of them will essentially lead to exactness 
of the approximation. It is reminded that (i) and (ii) can not be true. For (ii), a random 
variable with a symmetric distribution around 0, m = and EX^ — E[(X — m)-'] = for 
all odd j (and in particular j — 3,) but not for even j. 

(i). g"'{m) = implies that all other derivatives of order greater than three evaluated at 
X = m to be zero. In particular, g^'^\z) — for the random variable z lies between 
m and X. 
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(ii) . E[[X — m)^] = 0, implies that all other central moments of order greater than three 

about mean equal zero. 

(iii) . Remaining terms beyond third sum up to a zero. 

We use the following polynomial expansion later in the paper. The series expansion 
{x + y)" is symmetric with respect to the changes of variables x,y, so docs convergence 
region. The expression in parenthesis following of the series, indicates the region of 
convergence. If not otherwise indicated, it is to be understood that the series converges 
for all values of the variable. 

/ \« « n-i n(n—l) „_2 2 n(n — l)(n — 2) „_o , 122 
(x + y)" = + y + ^-^, — -x y +— -x"" Vh , where < x. 



III. EXACTNESS OF THE EXPECTED VALUE: 

In this section, attention is drawn to the conditions for which an exactness of the 
approximation can be achieved. 

Proposition 1 If g{-) is a polynomial of degree 2, then the statistical differential approx- 
imation for the expected value of the transformations of the random variable X, E[g{X)] 
is exact. 

The above proposition is extended to involve third degree polynomial in the following 
manner. 

Proposition 2 If g{-) is a polynomial of degree 3, then the statistical differential approx- 
imation for the expected value of the transformations of the random variable X, E[g{X)\ 
is exact, provided E[{X — m)^] = 0. 

The natural question is that could this procedure be so extended to the next higher 
degree polynomial by requiring vanishing fourth central moment about mean of the ran- 
dom variable, (in addition to what have been already assumed). A more generalized 
version of the above proposition is the following. 

Suppose g{-) is a polynomial of degree j, then requiring E[{X — m)^] = 0, for all 
3 < k < j will do the job! But, for A; = 4, this gives that E[{X - m)^] = 0, which 
immediately implies that we are dealing with a constant random variable, and there is no 
need to worry about anything else. 

The condition of the last assertion leads to a nice relation of the 3rd moment of the 
random variable X. 

Theorem 1 If X is a random variable such that its mean and variance exist and are 
known, together with the property, E[{X — m)^] = 0, then 3rd moment of the random 
variable X, E[X^] exists, is finite, and satisfies the relation. 
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Proof. 

Prom the series expansion for (x + y)"- with y'^ < x"^, and letting x — X — m, and 
y — m > 0, where E[X] — m, we obtain, 

= {X-my+j{X-my-^m+ ^^^ ~ ^\ x-my-^m'+ ^^^ ~ ~ ^\ x-my-^m^+- ■ ■ 

H — rri {X-m)m' H — (X-m)m' + m-^ . 

U-2)! U-1)! 

For j = 3, taking the expected value, and using E[{X - mf] = 0, we obtain, 

Var[X] 
iE[X]y 



E[X^]^iE[X]fU.^^ + l\, 



as asserted. 
I 

Our next task would be to see, are there other conditions for which this approximation 
is exact ? One of the results in this connection is to consider, the Peano Kernel method 

[5]; 

For any g e C"'~^^ [a, b] , the Taylor expansion with integral remainder gives, 
g{X) = f2 ^^^^r^9^^\a) + ^ [""{X- efg^''+^\e)de, where a<X<h, 

and g G C""'"^[a, 6] means g{-) is (n + 1) continuously differentiable function over [a,b]. 
Based on the above formula, we have: 

Theorem 2 If g{ ) has derivatives of order j < 3, g G C^[/], where I is some in- 
terval containing the range of X, and e(^J^{X — 9yg"'{0)d6^ = 0, then the statisti- 
cal differential approximations for the expected value of the transformations of the ran- 
dom variable X, E[g{X)] is exact. Otherwise, it will be exact up to the error term 

o{E{!^{x-efg"'{e)de)). 

Proof. 

The Peano Kernel method with n — 2, and a — m, gives 



(X-m) 

fc=0 

Taking expected values, we have 



E[Y] = E[g{X)] = g{m) + ^ • Var[X] + ^e{JjX - eyg"'{e)de). 

Now, the assertion of this theorem follows from the last equation. 
I 

The counter examples given in [4] have this condition satisfied. The theorem also 
provides us to extend the statistical differential approximation beyond the third term, by 
requiring appropriate number of central moments about mean. 
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Theorem 3 If g{ ) has derivatives of order j < n, g E C^^^[I], where I is some in- 
terval containing the range of X, and eQ^{X — Oy^ g^''^^^\6)d0^ = 0, then the sta- 
tistical differential approximations for the expected value of the transformations of the 
random variable X, E[g{X)] is exact. Otherwise, it will be exact up to the error term 

o{E(j^(x-9rg(-^'\e)de)). 

One of the short-coming of the last two theorems is that depending on the nature 
of the function g{-), the verification of this condition may be just as difhcuh as finding 
E[g{X)] in some cases. 

IV. EXACTNESS OF THE VARIANCE: 

By definition, 



Y8.T[Y] = E[[g{X)-E[g{X)^ 

Truncating the Taylor series for g{-), depending on the number of terms of the approxi- 
mating required, gives £"[1^], which then will be used to find Var g{X) . 
Suppose for an example, the expectation of g{X), 

E[Y]^g{m) + y'{m)-YsiT[X] 

is used, then the variance of g{X) is 

Var[y] = E[{Y - E[Y]Y] ^ E{g{X) - g{m) - ^g"{m) ■ Var[X]}'. 

Hence, the exactness of the variance formula still holds, if the function g{-), and the 
random variable X satisfy the condition stipulated in the theorems. 
Subject to first two terms of the expression for E[g{X)\, we have 

Var[y] = [^'(m)]^ ■ Var[X]. 

The applicable multivariate versions, involving covariance etc. given in [3] can also be 
derived in a similar manner. The necessary steps and conditions in deriving the approxi- 
mate expression for Var[y] are similar to those considered in the preceding discussion. 

V. ERROR BOUNDS: 

Since there are only few instances, where the approximation holds to be exact, we 
have no alternative, but to obtain some error bounds for this approximation. Then, it 
would be a question of deciding how small these bounds are. The following bounds are 
obtained, so that the accuracy of the approximations now entirely depend on the smallness 
of the error bounds, so desired. 

Some of the error bounds are computed for a class of functions, £, defined by 

Cg^{g:ge C-\m, h\ k \g^^>\-)\ < \gi-)\^^^ for aU j > 1}. 

This means that the derivatives are invariant under absolute value function. Most of the 
functions considered in the hteratures belong to this class. Note that g{x) = for a; > 0, 
and a > 0, does not belong to this class. 
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Theorem 4 If g{-) has all derivatives of order n > 1 such that |(?'-"'-'(-)| < \g{-)\^^\ for all 
j > 3, and e(^\X — m\^\g"'{X)\j exists, then the statistical differential approximations for 
the expected value of the transformations of the random variable X, E[g{X)] is exact up 
to the error term 0(e{{X - mf\g"'{X)\ 



Proof. 

From the Taylor expansion about mean, we have, 

iX - mf 

9{X) - g{m) -{X - m)g'(m) g'\m) 

= 3, 9 {to) + ^, g^'^'Km) + • • • + ^, g^-^\Ta) + • • • 

< (X- m)f i: (£z^i(/'(™))0)| + (£_^|r!|(5"'(„))<-3,| + . . I 

Using the fact that \g^^^{m)\ < \g{m)\^^\ we obtain, 

(2) < {X - + ^"^^^r ^'^^^^^"^^'^'""' + • • •} 

= (X-m)%'"(X)|. 

Taking the expected values, we obtain the required error bound as in the theorem. 
I 

The following corollary follows easily. 

Corollairy 1 Ifg{-) has bounded derivatives of ordern > 1, and E{\^X—m\^e^-^~'^^^ exists, 
then the statistical differential approximations for the expected value of the transformations 
of the random variable X, E[g[X)] is exact up to the error term O 

Proof. 

The proof of this corollary easily follows from the proof of the above theorem. 
I 

In the following, we use the monotonicity of the norm property of random variables, 
in a certain fashion, so as to find an error bound for this approximation. This is in fact, 
so called Lyapunov Inequality. For random variable X, 

{E[\X\']y'' > {EilXl'-y^', for all < r < s. 
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In our case, for random variable X ~ m, and < j < n, we have 

{E[\X - m|"]y/" > {E[\X - m\^}, for all < j < n. 

Thus, if we assume a particular higher absolute central moment about mean equals 
zero, then all other lower absolute central moments about mean will be zero. This case 
is not much interest to us as the approximation collapses to a trivial case. One of the 
required conditions, namely the condition (ii); E[{X — mY] = implies E[{X — m)''] = 0, 
for all > 3 has no rigorous impact in this case too. However, using the monotonicity 
property, we can establish an easy result. 

Theorem 5 If g{-) has all derivatives of order n > 1 such that g^\-) > 0, for alln > j > 
3, and E[\X — m\'^] exists for some integer n > 1, and letC — {E[\X — m\'^]}^^"' < oo, then 
the statistical differential approximations for the expected value of the transformations of 
the random variable X, E[g{X)] is exact up to the error term Oyg"'{C -\- m)^. 

Proof. 

From the Taylor expansion about mean, we have 

(X - mY 

9{X) - g{m) -{X - m)g'{m) H g"{m) 

= ^ g {rn) + ^ — —g^^^>{m) H h ^, g^ '{z), where z lies between m and h. 

Evaluating for the absolute values of the right side of the expression, by using properties 
of for all n > j > 3, and then taking the expected values, 

E[g{X)]-g{m)-]^g"{m)-V^r[X] 

< ^^^^^{g"'{m)r + ^^^^(/»)« + • • • + ^|^^(/'(.))("-^)}. 

Now, using Lyapunov Inequality, we have 



In summation notation, this equals to 

[j=o r- (n-3)! 

= {E|X-mrf/V"(C + m) 
= C'g"'{C + m)). 

This gives the required error bound having determined an interval containing z that is 
independent of the range of X. 
I 

Further, assuming that all derivatives of g{-) are bounded, an corollary is immediate. 
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Corollary 2 If g{-) has bounded derivatives of order n > 1, ami ev^H"^"™!"!'' exists for 
some integer n > 1, and is finite, then the statistical differential approximations for the 
expected value of the transformations of the random variable X, E[g{X)\ is exact up to 

the error term 0({E[\X - mn}^/"e(^[l^-"*l"l)'^"). 
Proof. 

The proof of this is essentially similar to the proof of the last theorem. 
I 

All of these results derived above can be extended for the statistical differential 
approximation truncated beyond third term under appropriate conditions. 

SUMMARY: 

This topic has been presented in a course on survival methods as a prelude to the 
other relevant chapters to follow, but failed to discuss the conditions under which the 
exactness of this approximation to hold. The lack of them, students would wonder is this 
approximation reasonable ?, and are also eager to find out the validity and accuracy of 
these results. A part of this discussion enriches rather subtle, and interesting topic, thus 
requiring inclusion in the future additions of [3]. Relevant rates of convergence, error 
analysis and similar results for other series expansions can be studied, if one needs to 
develop this topic for further research. 
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